Hierarchy motion estimation is usually adopted by high quality video codec in video compression as it can give desirable Rate-Distortion quality in different kinds of motion scenarios, especially in fast motion situation. First, it gets accurate full-pixel motion vector. In this stage, down sampling is used to generate low resolution sample, where full-pixel motion estimation is made to find the best motion vector of multi pixels. Hence, based on this result, full-pixel motion estimation in certain region of input raw video can get precise full-pixel motion vector. In the second stage of sub-pixel motion estimation, sub pixel interpolation is required, which introduces heavy computational burden. For example, in H.264, half-pixel interpolation requires 6-tap filter, and quarter-pixel interpolation needs 2-tap filter; whereas in H.265, more computation is needed as 8-tap filter and 7-tap filter are required. Because of highly-expensive interpolation, sub-pixel motion estimation occupies a significant amount of computation, sometimes even more than that of full pixel motion estimation.